Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CANN] Add doc and docker image #8867

Merged
merged 1 commit into from
Aug 19, 2024
Merged

[CANN] Add doc and docker image #8867

merged 1 commit into from
Aug 19, 2024

Conversation

wangshuai09
Copy link
Contributor

@github-actions github-actions bot added documentation Improvements or additions to documentation devops improvements to build systems and github actions labels Aug 5, 2024
@mofosyne mofosyne added the Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix label Aug 5, 2024
@xuedinge233
Copy link
Contributor

xuedinge233 commented Aug 7, 2024 •
edited
Loading

Here are same models in npu operation

model FP16 Q8_0 Q4_0
AquilaChat2-7B √ √ √
Baichuan-7b √ √ √
Baichuan2-7B-Chat √ √ √
bitnet_b1_58-large √ √ √
bloom-560m √ x √
bloomz-alpaca-560m √ x √
c4ai-command-r-35B-v01 x x x
chatglm3-6B x x x
chinese-alpaca-2-1.3b √ √ √
CodeShell-7B √ √ √
deepseek-ai_deepseek-coder-1.3B-base x x x
deepseek-ai_DeepSeek-V2-Lite x x x
deepseek-coder-6.7B-instruct x x x
DeepSeek-V2-Lite-64x1.5B x x x
falcon-7b-instruct √ √ √
flan-t5-large √ √ √
gemma-2-9b-it √ √ √
glm-4-9B x x x
gpt2 √ √ √
Gpt2-163M √ √ √
granite-3B-code-instruct √ √ √
GritLM-7B √ √ √
internlm2_5-7b-chat √ √ √
koala-7B-HF √ √ √
Llama-2-7b-chat-hf √ √ √
Llama-3-Smaug-8B √ √ √
Llama2-Chinese-7b-Chat √ √ √
Llama3-8B √ √ √
Llama3-8b-chinese √ √ √
mamba-130m-hf √ √ √
Mistral-7B-Instruct-v0.2 √ √ √
Mixtral-8x7B-Instruct-v0.1 X √ √
mpt-7B √ √ √
OLMo-1B-hf √ √ √
OpenELM-3B-Instruct √ √ √
Orion-14b-base √ √ √
phi1 x x x
phi2 x x x
Phi-3-mini-4k-instruct √ √ √
plamo-13b √ √ √
pythia-70M x x x
Qwen-7B √ √ √
Qwen2-1.5B-Instruct √ x √
Refact-1_6B-fim √ √ √
SmolLM-135M √ √ √
stablelm-zephyr x x x
stablelm-2-zephyr-1_6b x x x
starcoderbase-1b √ √ √
starcoder2-3b √ √ √
vigogne-7b-chat √ √ √
xverse-7b-chat √ √ √
Yi-6b-Chat √ √ √

@wangshuai09 wangshuai09 mentioned this pull request Aug 14, 2024
Open
@wangshuai09 wangshuai09 marked this pull request as ready for review August 15, 2024 07:31
@wangshuai09
Copy link
Contributor Author

@hipudding Please help review this work.

@hipudding hipudding added the Ascend NPU issues specific to Ascend NPUs label Aug 19, 2024
hipudding
hipudding reviewed Aug 19, 2024
View reviewed changes
docs/backend/CANN.md Outdated Show resolved Hide resolved
@hipudding hipudding requested review from slaren and ggerganov August 19, 2024 03:28
@wangshuai09 wangshuai09 force-pushed the doc_npu branch 3 times, most recently from dbc9d42 to d7b4983 Compare August 19, 2024 07:19
@wangshuai09 @xuedinge233 @hipudding
cann: add doc for cann backend
c109a90 
Co-authored-by: xuedinge233 <[email protected]>
Co-authored-by: hipudding <[email protected]>
@hipudding hipudding merged commit cfac111 into ggerganov:master Aug 19, 2024
7 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 15, 2024
@wangshuai09 @xuedinge233
@hipudding @arthw
cann: add doc for cann backend (ggerganov#8867)
7eac90f 
Co-authored-by: xuedinge233 <[email protected]>
Co-authored-by: hipudding <[email protected]>
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 18, 2024
@wangshuai09 @xuedinge233
@hipudding @arthw
cann: add doc for cann backend (ggerganov#8867)
1b64e0e 
Co-authored-by: xuedinge233 <[email protected]>
Co-authored-by: hipudding <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hipudding hipudding hipudding approved these changes

@ggerganov ggerganov ggerganov approved these changes

@slaren slaren Awaiting requested review from slaren

Assignees

@hipudding hipudding

@wangshuai09 wangshuai09

Labels
Ascend NPU issues specific to Ascend NPUs devops improvements to build systems and github actions documentation Improvements or additions to documentation Review Complexity : Low Trivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@wangshuai09 @xuedinge233 @ggerganov @hipudding @mofosyne